{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zwBCE43Cv3PH"
   },
   "source": [
    "## Tensorflow怎样保存与加载模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YQB7yiF6v9GR"
   },
   "source": [
    "#### 背景：模型训练和预估服务\n",
    "* 训练模型一般不是目标，在线预估服务才是\n",
    "* 一般会有一个程序训练模型，训练好之后把模型保存，在线预估加载模型实现服务\n",
    "* 模型一般包括模型结构、训练好的模型参数两部分组成\n",
    "\n",
    "#### 保存模型的方法\n",
    "* 模型结构+参数打包保存和加载\n",
    "* 模型结构存储到json文件，模型参数保存到h5文件\n",
    "* 模型结构存储到yaml文件，模型参数保存到h5文件\n",
    "\n",
    "其实，模型参数甚至也会存储到redis、mysql等，各个web服务可以随意更新加载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iiyC7HkqxlUD"
   },
   "source": [
    "### 1. 准备和训练一个模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5IoRbCA2n0_V"
   },
   "source": [
    "#### 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UEfJ8TcMpe-2"
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"./datas/heart/heart.csv\")\n",
    "\n",
    "# 把thal列变成数字编码\n",
    "df['thal'] = pd.Categorical(df['thal'])\n",
    "df['thal'] = df['thal'].cat.codes\n",
    "\n",
    "# 要预测的目标，这是个二分类问题\n",
    "target = df.pop('target')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LmCl5R5C2IKo"
   },
   "source": [
    "#### 构建dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "W6Yc-D3aqyBb"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:16:19.409425: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA\n",
      "To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2022-03-25 16:16:19.410072: I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 12. Tune using inter_op_parallelism_threads for best performance.\n"
     ]
    }
   ],
   "source": [
    "# 构建dataset，其实是把pandas数据转换成numpy数组进行转换的\n",
    "dataset = tf.data.Dataset.from_tensor_slices((df.values, target.values))\n",
    "# Shuffle and batch the dataset.\n",
    "train_dataset = dataset.shuffle(len(df)).batch(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 46.    0.    4.  138.  243.    0.    2.  152.    1.    0.    2.    0.\n",
      "    3. ]\n",
      " [ 55.    1.    4.  132.  353.    0.    0.  132.    1.    1.2   2.    1.\n",
      "    4. ]\n",
      " [ 49.    1.    3.  118.  149.    0.    2.  126.    0.    0.8   1.    3.\n",
      "    3. ]\n",
      " [ 53.    0.    4.  130.  264.    0.    2.  143.    0.    0.4   2.    0.\n",
      "    3. ]]\n"
     ]
    }
   ],
   "source": [
    "# 用于一会的测试\n",
    "for x, y in train_dataset.take(1):\n",
    "    input_data = x.numpy()\n",
    "    print(input_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "R3dQ-83Ztsgl"
   },
   "source": [
    "#### 搭建训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "76/76 [==============================] - 1s 12ms/step - loss: 1.4107 - accuracy: 0.7261\n",
      "Epoch 2/10\n",
      "76/76 [==============================] - 0s 2ms/step - loss: 1.0673 - accuracy: 0.7327\n",
      "Epoch 3/10\n",
      " 1/76 [..............................] - ETA: 0s - loss: 0.0068 - accuracy: 1.0000"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:16:20.372162: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n",
      "2022-03-25 16:16:20.525990: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "76/76 [==============================] - 0s 1ms/step - loss: 0.8690 - accuracy: 0.7426\n",
      "Epoch 4/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 0.8364 - accuracy: 0.7459\n",
      "Epoch 5/10\n",
      "45/76 [================>.............] - ETA: 0s - loss: 0.8009 - accuracy: 0.7667"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:16:20.631751: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n",
      "2022-03-25 16:16:20.735823: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "76/76 [==============================] - 0s 1ms/step - loss: 0.7381 - accuracy: 0.7756\n",
      "Epoch 6/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 0.6179 - accuracy: 0.7756\n",
      "Epoch 7/10\n",
      "44/76 [================>.............] - ETA: 0s - loss: 0.5196 - accuracy: 0.8182"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:16:20.835055: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n",
      "2022-03-25 16:16:20.937596: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "76/76 [==============================] - 0s 1ms/step - loss: 0.5462 - accuracy: 0.7954\n",
      "Epoch 8/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 0.5820 - accuracy: 0.8086\n",
      "Epoch 9/10\n",
      "43/76 [===============>..............] - ETA: 0s - loss: 0.5463 - accuracy: 0.7733"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:16:21.037740: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n",
      "2022-03-25 16:16:21.138065: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "76/76 [==============================] - 0s 1ms/step - loss: 0.5376 - accuracy: 0.7657\n",
      "Epoch 10/10\n",
      "76/76 [==============================] - 0s 1ms/step - loss: 0.4728 - accuracy: 0.7789\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-25 16:16:21.239884: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n",
      "2022-03-25 16:16:21.351845: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Out of range: End of sequence\n",
      "\t [[{{node IteratorGetNext}}]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f7a3d869e10>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = tf.keras.Sequential([\n",
    "    tf.keras.layers.Dense(10, input_shape=(df.shape[1],)),\n",
    "    tf.keras.layers.Dense(10, activation='relu'),\n",
    "    tf.keras.layers.Dense(1)\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "            loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
    "            metrics=['accuracy'])\n",
    "\n",
    "model.fit(train_dataset, epochs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iiyC7HkqxlUD"
   },
   "source": [
    "### 方法1：把模型结构和模型参数一起保存"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save(\"./models/heart_model_method1.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_total = tf.keras.models.load_model(\"./models/heart_model_method1.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4, 13)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 一个batch\n",
    "input_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.0199895 ],\n",
       "       [ 1.4073491 ],\n",
       "       [-0.06729454],\n",
       "       [-1.4674191 ]], dtype=float32)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_total.predict(input_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iiyC7HkqxlUD"
   },
   "source": [
    "### 方法2：模型结构保存到json，模型参数保存到h5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1434"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 将模型结构保存到json文件\n",
    "open(\"./models/heart_model_json.json\", \"w\").write(model.to_json())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将模型参数保存到.h5文件\n",
    "model.save_weights(\"./models/heart_model_json.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载模型\n",
    "model_json = tf.keras.models.model_from_json(open(\"./models/heart_model_json.json\").read())\n",
    "model_json.load_weights(\"./models/heart_model_json.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 注意需要重新编译模型\n",
    "model_json.compile(optimizer='adam',\n",
    "            loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
    "            metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.0199895 ],\n",
       "       [ 1.4073491 ],\n",
       "       [-0.06729454],\n",
       "       [-1.4674191 ]], dtype=float32)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 实现预估\n",
    "model_json.predict(input_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iiyC7HkqxlUD"
   },
   "source": [
    "### 方法3：模型结构保存到yaml，模型参数保存到h5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1591"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 将模型结构保存到yaml文件\n",
    "open(\"./models/heart_model_yaml.yaml\", \"w\").write(model.to_yaml())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将模型参数保存到.h5文件\n",
    "model.save_weights(\"./models/heart_model_yaml.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/peishuaishuai/opt/anaconda3/envs/pytensorflow/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/model_config.py:76: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.\n",
      "  config = yaml.load(yaml_string)\n"
     ]
    }
   ],
   "source": [
    "# 加载模型\n",
    "model_yaml = tf.keras.models.model_from_yaml(open(\"./models/heart_model_yaml.yaml\").read())\n",
    "model_yaml.load_weights(\"./models/heart_model_yaml.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 注意需要重新编译模型\n",
    "model_yaml.compile(optimizer='adam',\n",
    "            loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
    "            metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.0199895 ],\n",
       "       [ 1.4073491 ],\n",
       "       [-0.06729454],\n",
       "       [-1.4674191 ]], dtype=float32)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 实现预估\n",
    "model_yaml.predict(input_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 46. ,   0. ,   4. , 138. , 243. ,   0. ,   2. , 152. ,   1. ,\n",
       "          0. ,   2. ,   0. ,   3. ],\n",
       "       [ 55. ,   1. ,   4. , 132. , 353. ,   0. ,   0. , 132. ,   1. ,\n",
       "          1.2,   2. ,   1. ,   4. ],\n",
       "       [ 49. ,   1. ,   3. , 118. , 149. ,   0. ,   2. , 126. ,   0. ,\n",
       "          0.8,   1. ,   3. ,   3. ],\n",
       "       [ 53. ,   0. ,   4. , 130. , 264. ,   0. ,   2. , 143. ,   0. ,\n",
       "          0.4,   2. ,   0. ,   3. ]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'[[46.0,0.0,4.0,138.0,243.0,0.0,2.0,152.0,1.0,0.0,2.0,0.0,3.0],[55.0,1.0,4.0,132.0,353.0,0.0,0.0,132.0,1.0,1.2,2.0,1.0,4.0],[49.0,1.0,3.0,118.0,149.0,0.0,2.0,126.0,0.0,0.8,1.0,3.0,3.0],[53.0,0.0,4.0,130.0,264.0,0.0,2.0,143.0,0.0,0.4,2.0,0.0,3.0]]'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "pd.DataFrame(input_data).to_json(orient='values')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "pandas_dataframe.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
